Variant Of Genetic Programming Research Articles

An ensemble learning interpretation of geometric semantic genetic programming

AbstractGeometric semantic genetic programming (GSGP) is a variant of genetic programming (GP) that directly searches the semantic space of programs to produce candidate solutions. GSGP has shown considerable success in improving the performance of GP in terms of program correctness, however this comes at the expense of exponential program growth. Subsequent attempts to address this growth have not fully-exploited the fact that GSGP searches by producing linear combinations of existing solutions. This paper examines this property of GSGP and frames the method as an ensemble learning approach by redefining mutation and crossover as examples of boosting and stacking, respectively. The ensemble interpretation allows for simple integration of regularisation techniques that significantly reduce the size of the resultant programs. Additionally, this paper examines the quality of parse tree base learners within this ensemble learning interpretation of GSGP and suggests that future research could substantially improve the quality of GSGP by examining more effective initialisation techniques. The resulting ensemble learning interpretation leads to variants of GSGP that substantially improve upon the performance of traditional GSGP in regression contexts, and produce a method that frequently outperforms gradient boosting.

Cellular geometric semantic genetic programming

Among the different variants of Genetic Programming (GP), Geometric Semantic GP (GSGP) has proved to be both efficient and effective in finding good solutions. The fact that the operators of GSGP operate on the semantics of the individuals in a clear way provides guarantees on the way the search is performed. GSGP is not, however, free from limitations like the premature convergence of the population to a small–and possibly sub-optimal–area of the search space. One reason for this issue could be the fact that good individuals can quickly “spread” in the population suppressing the emergence of competition. To mitigate this problem, we impose a cellular automata (CA) inspired communication topology over GSGP. In CAs a collection of agents (as finite state automata) are positioned in a n-dimensional periodic grid and communicates only locally with the automata in their neighbourhoods. Similarly, we assign a location to each individual on an n-dimensional grid and the entire evolution for an individual will happen locally by considering, for each individual, only the individuals in its neighbourhood. Specifically, we present an algorithm in which, for each generation, a subset of the neighbourhood of each individual is sampled and the selection for the given cell in the grid is performed by extracting the two best individuals of this subset, which are employed as parents for the Geometric Semantic Crossover. We compare this cellular GSGP (cGSGP) approach with standard GSGP on eight regression problems, showing that it can provide better solutions than GSGP. Moreover, by analyzing convergence rates, we show that the improvement is observable regardless of the number of executed generations. As a side effect, we additionally show that combining a small-neighbourhood-based cellular spatial structure with GSGP helps in producing smaller solutions. Finally, we measure the spatial autocorrelation of the population by adopting the Moran’s I coefficient to provide an overview of the diversity, showing that our cellular spatial structure helps in providing better diversity during the early stages of the evolution.

Evolving Multi-Output Digital Circuits Using Multi-Genome Grammatical Evolution

Grammatical Evolution is a Genetic Programming variant which evolves problems in any arbitrary language that is BNF compliant. Since its inception, Grammatical Evolution has been used to solve real-world problems in different domains such as bio-informatics, architecture design, financial modelling, music, software testing, game artificial intelligence and parallel programming. Multi-output problems deal with predicting numerous output variables simultaneously, a notoriously difficult problem. We present a Multi-Genome Grammatical Evolution better suited for tackling multi-output problems, specifically digital circuits. The Multi-Genome consists of multiple genomes, each evolving a solution to a single unique output variable. Each genome is mapped to create its executable object. The mapping mechanism, genetic, selection, and replacement operators have been adapted to make them well-suited for the Multi-Genome representation and the implementation of a new wrapping operator. Additionally, custom grammar syntax rules and a cyclic dependency-checking algorithm have been presented to facilitate the evolution of inter-output dependencies which may exist in multi-output problems. Multi-Genome Grammatical Evolution is tested on combinational digital circuit benchmark problems. Results show Multi-Genome Grammatical Evolution performs significantly better than standard Grammatical Evolution on these benchmark problems.

Soft-Computing-Based Estimation of a Static Load for an Overhead Crane.

Payload weight detection plays an important role in condition monitoring and automation of cranes. Crane cells and scales are commonly used in industrial practice; however, when their installation to the hoisting equipment is not possible or costly, an alternative solution is to derive information about the load weight indirectly from other sensors. In this paper, a static payload weight is estimated based on the local strain of a crane's girder and the current position of the trolley. Soft-computing-based techniques are used to derive a nonlinear input-output relationship between the measured signals and the estimated payload mass. Data-driven identification is performed using a novel variant of genetic programming named grammar-guided genetic programming with sparse regression, multi-gene genetic programming, and subtractive fuzzy clustering method combined with the least squares algorithm on experimental data obtained from a laboratory overhead crane. A comparative analysis of the methods showed that multi-gene genetic programming and grammar-guided genetic programming with sparse regression performed similarly in terms of accuracy and both performed better than subtractive fuzzy clustering. The novel approach was able to find a more parsimonious model with its direct implantation having a lower execution time.

A novel binary classification approach based on geometric semantic genetic programming

Geometric semantic genetic programming (GSGP) is a recent variant of genetic programming. GSGP allows the landscape of any supervised regression problem to be transformed into a unimodal error surface, thus it has been applied only to this kind of problem. In a previous paper, we presented a novel variant of GSGP for binary classification problems that, taking inspiration from perceptron neural networks, uses a logistic-based activation function to constrain the output value of a GSGP tree in the interval [0,1]. This simple approach allowed us to use the standard RMSE function to evaluate the train classification error on binary classification problems and, consequently, to preserve the intrinsic properties of the geometric semantic operators. The results encouraged us to investigate this approach further. To this aim, in this paper, we present the results from 18 test problems, which we compared with those achieved by eleven well-known and widely classification schemes. We also studied how the parameter settings affect the classification performance and the use of the F-score function to deal with imbalanced data. The results confirmed the effectiveness of the proposed approach.

GP-DMD: a genetic programming variant with dynamic management of diversity

The proper management of diversity is essential to the success of Evolutionary Algorithms. Specifically, methods that explicitly relate the amount of diversity maintained in the population to the stopping criterion and elapsed period of execution, with the aim of attaining a gradual shift from exploration to exploitation, have been particularly successful. However, in the area of Genetic Programming, the performance of this design principle has not been studied. In this paper, a novel Genetic Programming method, Genetic Programming with Dynamic Management of Diversity (GP-DMD), is presented. GP-DMD applies this design principle through a replacement strategy that combines penalties based on distance-like functions with a multi-objective Pareto selection based on accuracy and simplicity. The proposed general method was adapted to the well-established Symbolic Regression benchmark problem using tree-based Genetic Programming. Several state-of-the-art diversity management approaches were considered for the experimental validation, and the results obtained showcase the improvements both in terms of mean square error and size. The effects of GP-DMD on the dynamics of the population are also analyzed, revealing the reasons for its superiority. As in other fields of Evolutionary Computation, this design principle contributes significantly to the area of Genetic Programming.

Towards a Vectorial Approach to Predict Beef Farm Performance

Accurate livestock management can be achieved by means of predictive models. Critical factors affecting the welfare of intensive beef cattle husbandry systems can be difficult to be detected, and Machine Learning appears as a promising approach to investigate the hundreds of variables and temporal patterns lying in the data. In this article, we explore the use of Genetic Programming (GP) to build a predictive model for the performance of Piemontese beef cattle farms. In particular, we investigate the use of vectorial GP, a recently developed variant of GP, that is particularly suitable to manage data in a vectorial form. The experiments conducted on the data from 2014 to 2018 confirm that vectorial GP can outperform not only the standard version of GP but also a number of state-of-the-art Machine Learning methods, such as k-Nearest Neighbors, Generalized Linear Models, feed-forward Neural Networks, and long- and short-term memory Recurrent Neural Networks, both in terms of accuracy and generalizability. Moreover, the intrinsic ability of GP in performing an automatic feature selection, while generating interpretable predictive models, allows highlighting the main elements influencing the breeding performance.

Mine subsidence prediction using gene expression programming based on multivariable symbolic regression

Accurate prediction of surface subsidence becomes a significant challenge for active industrial companies in coal mining fields due to the importance of the economic impacts of longwall mining-induced subsidence. This article explores a new variant of genetic programming, namely gene expression programming (GEP). The GEP-based method is utilized to present a new mathematical formula for subsidence prediction in longwall coal mining. The derived model includes both geometrical and geological variables. The data set consists of field measurements obtained through 37 longwall panels of Ulan coal mine, NSW, Australia. The GEP-based model concluded satisfactory subsidence prediction outcomes compared to other empirical methods such as NCB, DMR, ACARP, and IPM. The predictive ability of the GEP-based models, which captures the complex nonlinear effects of the critical factors on the magnitude of subsidence, resulted in a statistically significant improvement in predictive capacity compared to the aforementioned empirical methods. The sensitivity analysis results indicated that Panel width and cover thickness with 31% and 23% were the most influential parameters in the proposed model. Also, the extracted seam thickness, thick layer location, and thick layer thickness had 19%, 16%, and 11% impact on the GEP proposed model, respectively.

FMCGP: frameshift mutation cartesian genetic programming

Cartesian Genetic Programming (CGP) is a variant of Genetic Programming (GP) with the individuals represented by a two-dimensional acyclic directed graph, which can flexibly encode many computing structures. In general, CGP only uses a point mutation operator and the genotype of an individual is of fixed size, which may lead to the lack of population diversity and then cause the premature convergence. To address this problem in CGP, we propose a Frameshift Mutation Cartesian Genetic Programming (FMCGP), which is inspired by the DNA mutation mechanism in biology and the frameshift mutation caused by insertion or deletion of nodes is introduced to CGP. The individual in FMCGP has variable-length genotype and the proposed frameshift mutation operator helps to generate more diverse offspring individuals by changing the compiling framework of genotype. FMCGP is evaluated on the symbolic regression problems and Even-parity problems. Experimental results show that FMCGP does not exhibit the bloat problem and the use of frameshift mutation improves the search performance of the standard CGP.

Regresión Simbólica aplicada a la Predicción del Consumo Eléctrico a Corto Plazo en el Nivel de Subestación

El modelado de datos es un problema fundamental en diversas áreas del conocimiento. La Regresión Simbólica es una técnica que permite encontrar una relación matemática para describir un conjunto de datos experimentales. A diferencia de los métodos tradicionales de modelado, la Programación Genética permite encontrar una expresión matemática susceptible de ser analizada e interpretada. Multi-Expression Programming es una variante de Programación Genética, que presenta múltiples ventajas, haciéndola apta para su uso en casos reales. En este trabajo proponemos aplicar dicha variante para descubrir modelos de predicción, un día hacia adelante, del consumo eléctrico de subestación transformadora en Tucumán, Argentina. Para analizar el comportamiento del algoritmo y ajustar parámetros, se realizaron pruebas utilizando Benchmarks conocidos. Se concluye que Multi-Expression Programming es adecuada para encontrar modelos en problemas complejos, y en el caso de predicción de consumo eléctrico se logró un nivel de error similar al obtenido con otras técnicas.

A benchmark suite for designing combinational logic circuits via metaheuristics

The evolvable hardware literature reports several methods for the evolution of digital circuits. However, there is a large variability in the set of problems and the appropriate metrics used for the evaluation of the results. In this paper, we propose a set of representative problems to comparatively evaluate metaheuristics when designing Combinational Logic Circuits (CLCs). We also define a set of performance measurements and descriptive statistics to analyze the results found by the search techniques. As a case study, we compare Cartesian Genetic Programming variants applied to this domain. The results highlight the benefits of the proposed heterogeneous benchmark suite in the analysis of metaheuristics when designing CLCs.

Learning feature spaces for regression with genetic programming.

Genetic programming has found recent success as a tool for learning sets of features for regression and classification. Multidimensional genetic programming is a useful variant of genetic programming for this task because it represents candidate solutions as sets of programs. These sets of programs expose additional information that can be exploited for building block identification. In this work, we discuss this architecture and others in terms of their propensity for allowing heuristic search to utilize information during the evolutionary process. We investigate methods for biasing the components of programs that are promoted in order to guide search towards useful and complementary feature spaces. We study two main approaches: 1) the introduction of new objectives and 2) the use of specialized semantic variation operators. We find that a semantic crossover operator based on stagewise regression leads to significant improvements on a set of regression problems. The inclusion of semantic crossover produces state-of-the-art results in a large benchmark study of open-source regression problems in comparison to several state-of-the-art machine learning approaches and other genetic programming frameworks. Finally, we look at the collinearity and complexity of the data representations produced by different methods, in order to assess whether relevant, concise, and independent factors of variation can be produced in application.

A Preliminary Study of Knowledge Transfer in Multi-Classification Using Gene Expression Programming.

Gene Expression Programming (GEP), a variant of Genetic Programming (GP), is a well established technique for automatic generation of computer programs. Due to the flexible representation, GEP has long been concerned as a classification algorithm for various applications. Whereas, GEP cannot be extended to multi-classification directly, and thus is only capable of treating an M-classification task as M separate binary classifications without considering the inter-relationship among classes. Consequently, GEP-based multi-classifier may suffer from output conflict of various class labels, and the underlying conflict can probably lead to the degraded performance in multi-classification. This paper employs evolutionary multitasking optimization paradigm in an existing GEP-based multi-classification framework, so as to alleviate the output conflict of each separate binary GEP classifier. Therefore, several knowledge transfer strategies are implemented to enable the interation among the population of each separate binary task. Experimental results on 10 high-dimensional datasets indicate that knowledge transfer among separate binary classifiers can enhance multi-classification performance within the same computational budget.

State-of-art of genetic programming applications in water-resources systems analysis.

Evolutionary algorithms (EAs) have become competitive solvers of a wide variety of water-resources optimization problems. Genetic programming (GP) has become a leading EA since its inception in 1985. This paper reviews the state-of-the-art of GP and its applications in water-resources systems analysis. A comprehensive knowledge about GP's theory and modeling approach is essential for its successful application in water-resources systems analysis. This review presents variants of GP that have been proven useful in various applications to water resources problems. Several examples of applications of GP in water-resources systems analysis are herein presented. This review reveals GP's capability and superiority compared to other conventional methods, which makes it suitable for solving a wide variety of water-related problems including rainfall-runoff modeling, streamflow sediment prediction, flood prediction and routing, evaporation and evapotranspiration forecasting, reservoir operation, groundwater modeling, water quality modeling, water demand forecasting, and water distribution systems.

Parameter identification for symbolic regression using nonlinear least squares

In this paper we analyze the effects of using nonlinear least squares for parameter identification of symbolic regression models and integrate it as local search mechanism in tree-based genetic programming. We employ the Levenberg–Marquardt algorithm for parameter optimization and calculate gradients via automatic differentiation. We provide examples where the parameter identification succeeds and fails and highlight its computational overhead. Using an extensive suite of symbolic regression benchmark problems we demonstrate the increased performance when incorporating nonlinear least squares within genetic programming. Our results are compared with recently published results obtained by several genetic programming variants and state of the art machine learning algorithms. Genetic programming with nonlinear least squares performs among the best on the defined benchmark suite and the local search can be easily integrated in different genetic programming algorithms as long as only differentiable functions are used within the models.

Multi Gene Genetic Program Modelling on L-Asparaginase Activity of Bacillus Stratosphericus

Abstract The current study focuses on maximization of L-Asparaginase production from Bacillus stratosphericus isolated from Ocimum tenuiflorum. Optimization study followed by modelling using Artificial Neural Network (ANN) was performed. The experimental data obtained from Response Surface Methodology (RSM) was further studied by an evolutionary algorithm Genetic Programming (GP) to find the prediction equation. GP does not require prior knowledge of the data sets. GP is an extension of Genetic Algorithm (GA), where the results are represented in the form of trees. Multi gene genetic programming (MGPP) is a variant of GP used to solve non-linear mathematical models. The prediction equation obtained from the GP analysis is represented in the form of tree. Each tree represents single gene. Best fit individuals obtained at each generation by using genetic operators were selected to get better regression co-efficient value. The predicted and experimental data showed good significance with R2 = 0.99956.

Recent Developments in Cartesian Genetic Programming and its Variants

Cartesian Genetic Programming (CGP) is a variant of Genetic Programming with several advantages. During the last one and a half decades, CGP has been further extended to several other forms with lots of promising advantages and applications. This article formally discusses the classical form of CGP and its six different variants proposed so far, which include Embedded CGP, Self-Modifying CGP, Recurrent CGP, Mixed-Type CGP, Balanced CGP, and Differential CGP. Also, this article makes a comparison among these variants in terms of population representations, various constraints in representation, operators and functions applied, and algorithms used. Further, future work directions and open problems in the area have been discussed.

State-of-the-art predictive modeling of TBM performance in changing geological conditions through gene expression programming

Hard rock TBM performance prediction is of great interest to the tunneling community on account of its importance in time and cost risk management of underground projects. Continuous development of new empirical models in recent decades reveals the importance of accurate prediction of this factor in diverse ground and machine conditions. The great number of different parameters influencing TBM performance and the high variability linked to specific field conditions cause the problem to be very complex. Gene expression programming (GEP) models, a robust variant of genetic programming, are developed in this study to correlate hard rock TBM performance with routine ground properties for project design applications. The developed models are compared with those from statistical and soft computing-based models in the literature. Overall, GEP models show good performance and are proven to be much better than the previous models. The proposed models of this study can be remarked as an ultimate stage to one decade of researchers’ attempts to improve the accuracy of predictive equations developed through a well-known database of TBM performance in one of the most complex tunneling projects in the world.

How Am I Driving? Using Genetic Programming to Generate Scoring Functions for Urban Driving Behavior

Road traffic injuries are a serious concern in emerging economies. Their death toll and economic impact are shocking, with 9 out of 10 deaths occurring in low or middle-income countries; and road traffic crashes representing 3% of their gross domestic product. One way to mitigate these issues is to develop technology to effectively assist the driver, perhaps making him more aware about how her (his) decisions influence safety. Following this idea, in this paper we evaluate computational models that can score the behavior of a driver based on a risky-safety scale. Potential applications of these models include car rental agencies, insurance companies or transportation service providers. In a previous work, we showed that Genetic Programming (GP) was a successful methodology to evolve mathematical functions with the ability to learn how people subjectively score a road trip. The input to this model was a vector of frequencies of risky maneuvers, which were supposed to be detected in a sensor layer. Moreover, GP was shown, even with statistical significance, to be better than six other Machine Learning strategies, including Neural Networks, Support Vector Regression and a Fuzzy Inference system, among others. A pending task, since then, was to evaluate if a more detailed comparison of different strategies based on GP could improve upon the best GP model. In this work, we evaluate, side by side, scoring functions evolved by three different variants of GP. In the end, the results suggest that two of these strategies are very competitive in terms of accuracy and simplicity, both generating models that could be implemented in current technology that seeks to assist the driver in real-world scenarios.

Forecasting performance of regional innovation systems using semantic-based genetic programming with local search optimizer

Innovation performance of regional innovation systems can serve as an important tool for policymaking to identify best practices and provide aid to regions in need. Accurate forecasting of regional innovation performance plays a critical role in the implementation of policies intended to support innovation because it can be used to simulate the effects of actions and strategies. However, innovation is a complex and dynamic socio-economic phenomenon. Moreover, patterns in regional innovation structures are becoming increasingly diverse and non-linear. Therefore, to develop an accurate forecasting tool for this problem represents a challenge for optimization methods. The main aim of the paper is to develop a model based on a variant of genetic programming to address the regional innovation performance forecasting problem. Using the historical data related to regional knowledge base and competitiveness, the model should accurately and effectively predict a variety of innovation outputs, including patent counts, technological and non-technological innovation activity and economic effects of innovations. We show that the proposed model outperforms state-of-the-art machine learning methods.